Algorithms for Data Science : Lecture 3

1 Concentration Inequalities Lemma 1 (Markov’s inequality). Let X be a non-negative random variable. For all λ > 0, Pr[X > λ] ≤ E[X] λ Lemma 2 (Chebyshev Inequality). For all λ > 0, Pr[|X −E[X]| > λ] ≤ var[X] λ2 Lemma 3 (The Chernoff Bound: Upper bound). Let X1, X2, ..., Xn be independent random variables taking values in {0, 1} with E[Xi] = pi. Let X = ∑n i=1Xi, and μ = E[X]. Then the following holds 1. For any δ > 0, Pr[X ≥ (1 + δ)μ] < ( eδ (1 + δ)(1+δ) )μ